Basil: A Fast and Byzantine-Resilient Approach for Decentralized Training

Detection and mitigation of Byzantine behaviors in a decentralized learning setting is daunting task, especially when the data distribution at users heterogeneous. As our main contribution, we propose Basil, fast computationally efficient robust algorithm for training systems, which leverages novel sequential, memory assisted performance-based criteria over logical ring while filtering users. In IID dataset setting, provide theoretical convergence guarantees demonstrating its linear rate. Furthermore, experimentally demonstrate that Basil to various attacks, including strong Hidden attack, providing up ${\sim}16 \%$ higher test accuracy state-of-the-art Byzantine-resilient approach. Additionally, generalize non-IID by proposing Anonymous Cyclic Data Sharing (ACDS), technique allows each node anonymously share random fraction local non-sensitive (e.g., landmarks images) with all other nodes. We alongside ACDS only $5\%$ sharing provides effective toleration nodes, unlike completely fails heterogeneous setting. Finally, reduce overall latency resulting from sequential implementation ring, Basil+. particular, Basil+ scalability enabling Byzantine-robust parallel across groups rings, same time, it retains performance gains due within group. through different sets experiments.